Training Ternary Neural Networks with Exact Proximal Operator
نویسندگان
چکیده
In this paper, we propose a stochastic proximal gradient method to train ternary weight neural networks (TNN). The proposed method features weight ternarization via an exact formula of proximal operator. Our experiments show that our trained TNN are able to preserve the state-of-the-art performance on MNIST and CIFAR-10 benchmark datesets.
منابع مشابه
The Prediction of Surface Tension of Ternary Mixtures at Different Temperatures Using Artificial Neural Networks
In this work, artificial neural network (ANN) has been employed to propose a practical model for predicting the surface tension of multi-component mixtures. In order to develop a reliable model based on the ANN, a comprehensive experimental data set including 15 ternary liquid mixtures at different temperatures was employed. These systems consist of 777 data points generally containing hydrocar...
متن کاملNavigation of a Mobile Robot Using Virtual Potential Field and Artificial Neural Network
Mobile robot navigation is one of the basic problems in robotics. In this paper, a new approach is proposed for autonomous mobile robot navigation in an unknown environment. The proposed approach is based on learning virtual parallel paths that propel the mobile robot toward the track using a multi-layer, feed-forward neural network. For training, a human operator navigates the mobile robot in ...
متن کاملCompressing Low Precision Deep Neural Networks Using Sparsity-Induced Regularization in Ternary Networks
A low precision deep neural network training technique for producing sparse, ternary neural networks is presented. The technique incorporates hardware implementation costs during training to achieve significant model compression for inference. Training involves three stages: network training using L2 regularization and a quantization threshold regularizer, quantization pruning, and finally retr...
متن کاملA constructive training algorithm for feedforward neural networks with ternary weights
We address the problem of training a special kind of Boolean feedforward neural networks, namely networks built from linear binary threshold units with ternary weights called majority units. We propose an original constructive training algorithm, inspired from both the approaches of the Tiling and the Upstart algorithms, and we give a simple proof of its convergence. Numerical experiments were ...
متن کاملA Differential Evolution and Spatial Distribution based Local Search for Training Fuzzy Wavelet Neural Network
Abstract Many parameter-tuning algorithms have been proposed for training Fuzzy Wavelet Neural Networks (FWNNs). Absence of appropriate structure, convergence to local optima and low speed in learning algorithms are deficiencies of FWNNs in previous studies. In this paper, a Memetic Algorithm (MA) is introduced to train FWNN for addressing aforementioned learning lacks. Differential Evolution...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- CoRR
دوره abs/1612.06052 شماره
صفحات -
تاریخ انتشار 2016